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DEFENDANTS-APPELLANTS’ BRIEF 


Preliminary Statement 


The defendants appeal from an Order of the United 
States District Court for the Southern Distriet of New 
York (Lasker, J.), which (1) declared constitutionally 
invalid five civil service examinations out of a series of 


nine examinations which were given for positions in the 


New York City Human Resources Administration; (2) 
permanently enjoined defendants from using the results 


of the examinations for 
for making permanent or 
positions in the Human 
directed defendants ti 
selection procedures 


any purpose, and, specifically, 
provisionsal appointments to 
Resources Administr: tion; (3) 
» develop lawful non-discriminatory 
for the positions at issue. and to 


validate these procedures in accordance with th EEOC 


Guidelines on Employment Selection Procedures, as those 
revised; and (4) required 
defendants to submit to the Court within 
date of the 


Guidelines are now or as later 


80 days of thi 
the development of 
approval and to fur- 
plaintiffs for comments on its 


order a detailed plan for 
¢tion procedures for its 
nish a copy of the plan to 
propriety, 


the new 


Plaintiffs appeal from that portion of the Order which 
denied their request for an award of attorneys’ fe 


This brief | 


resents the City defendants’ 
Appeal. 


points on their 


Question Pre -=nted 


This ease follows in the wake of those successfully 


challenging on constitutional grounds the selection 


pr ) 
sovernmental units to make appoint 
positions on the 
fitness. In order to succeed 
tiffs must first establish 
crimination. This 
Statisties 


cedures employ ed by 


ments to civil service basis of merit and 


in cases of this kind, plain 
a prima faci case of racial dis 
is customarily done by introducing 
Which demonsirate that Blacks and 
have fared significantly worse than 
procedure involved, which, 


Hispanies 
whites on the selection 
in civil service Jurisdictions, 
is normally a written examination. 


The role of statistics in these 
by professionally accepted stat 
examinations had 


cases is to demonstrate. 


istical techniques, that the 
a disproportionately adverse 


impact on 


the minority candidst taking the examination whi 


would not have occurred as a matter of chance, unrelated 
to race. The disparity must be of “significant magnitude 
to amount to a prona fi case of invidious discrimina 
tion”. Chance . LB ard of Kavam WETS, $58 F.2d 9 
1176 (2d Cir. 1972). 


It is our contention that the District Judge was clearly 


t 


erroneous as a n.etter ol 


invomplete figures relied upon by p‘aintiffs, from which 


law in holding that substantially 


no statistical conclusions could rroperly be drawn, was 


sufficient to establish a pro acie case of racial dis 


| 1 


‘rimination on the minorities as a whole who took these 


examinations. 


Once the prina facie cas established, a heavy bur 
den shifts to the defendants to ] ve that the examination 
was job-related to the position for which they were given. 


The preliminary focus is on the method of preparation 
of the examination. Careful preparation gives eround 
for an inference, rebuttable to be sure, that success was 
achieved in arriving at a valid examination. Vulcan So- 
ciety v. Civil Service Commis n, 490 F.2d 387, .396 (2d 
Cir. 1973). Here, there was evidence of eareful and 


‘ 


conscientious preparation 0 the eXaminat t 


ons al issue, 
but defendants derived no benefit from their efforts, let 
alone a rebuttable presumption o ccess. The District 
Court discounted the method of p  , aration entirely on 
the basis of testimony of the expert produced by plain 
tiffs who found numerous grounds for criticizing the 


preparation of the e amin: ‘ions. 


Regardless of deficiencies in the method of prepara 
pre} 


tion of the examinations, the task of the Court was to 
determine if the examinations were job related to the po- 
sitions at issue. This is a factual determination made by 


evaluating the examinations in light of the duties and 


responsibilities of the sitions for which they te 1. 
Ifere, there is no indication from the opinion that “he 
Court evaluated the examinations at all. Indeed, it had 
no need to evaluate the examinations since it had alrea ly 
decided against defendants on the basis of alleged defi- 
ciencies in the job analysis before it gave serious con 
sideration to the positions being tested for and the pur- 
pose for which the examinations were given. 

It is ovr contention that the Court below clearly erred 
n finding that defendants’ method of test preparation 
was inconsistent with professionally acceptable standards 
of test preparation and, believing his inquiry ended at 
that point, refusing to evaluate the product of that prep- 
aration, the examinations themselves. 


Facts 


These are two civil rights action . consolidated for all 
purposes, brought under 42 U.S.C. ¢¢ 1981 and 1983 by 


minority applicants on behalf of a class of such persons 


who failed civil service examinations given for positions 
in the Hfuman Resources Administration.* The Jones 
plaintiffs instituted suit on or about September 5, 1973, 
seeking preliminary injunctive relief restrainine defend 
ants from making permanent appointments and from ter 
minating minority provisionals for purposes of replacing 


them with persons from the eligible lists established as 


* Plaintiffs sought to include within the class those minorities 
who had scored too low to be appointed. Cf. Vulcan Society \ 
Civil Service Commission, 360 F. Supp. 1265, 1266 (S.D.N.Y 
i973). At first, the District Court denied the motion for a class 
action entirely (365a); then granted the motion as plaintiffs re 


questec (375a); then modified the class designation (377a) 


tions at issue 
were both the promotional (No. 1651) and the open com 
petitive (No. 2013) examinations for the position of Su 
pervising Human Resources Specialist (“Sup TRS") in 
the City’s Human Resources Administ? nm. The com- 
plaint alleged that the examinations ] a disparate im- 
pact on minorities and were job-related to the position 
of Sup HRS. 


The appointments were scheduled to he made for Sep- 
tember 14, 1973, and on that date t Court entered an 


‘ a trial on the 


he 
Order enjoining appointments pendin: 
merits (94a-96a).* 


The Williams complaint was served on January 4, 1974, 
three days before trial was to begin in Jones. At issue 


here were three examinations for two lower positions in 
the Iluman Resources occupational series given the same 
day as those in Jones. They were the promotional (No. 
1626) and open competitive . 1099) examinations for 
Senior Human Resources alist (“Sen HRS”) and 
the open competitive (No. 1!° ) examination for Human 


Resources Specialist (“hi thes In view of the identity 


of issues and proof in the two eases, they were consoli- 


*In light of the affidavit of Harry I. Bronstein, then Chair- 


man of the New York City Civil Service Commission and Director 


( the City Department of Personnel (80a), the Court en- 
joined as well a forthcoming promotional exam for the highest 


dated for all purposes, and an injunction against appoint- 
ments and termination of provisionals similar to that in 
Jones issued (188a-189a). 


The trial on the merits lasted four days api was con- 
‘ined to the two Sup HRS exams. Plaintiffs introduced 
the raw figures supplied to them by defendants as proof 
that the examinations had a disparate effect upon minor- 
ities. That data is set forth in Table T at the end of 
defendants’ brief. The ficures showed that for the pro- 
motional Sup HRS exam (No. 1631), 138 persons took 
the exam. Of the 43 persons who passed the exam: 28 
were White, 12 Black and 3 Hispanic. The pass rates 
for these groups were respectively 54%, 17% and 16%. 
For the open competitive Sup HRS exam (No. 2013), 


994 took the examination. Sine this was an open com- 


petitive exam, defendants had information en the ethnic 
identity and pass-rates only of current employees of 
HRA. Of those identifiable, 125 Whites, or 59% 


2: passed, 
compared to 39 Blacks (16%), and 3 ispanies (15%). 
The racial identity and corresponding pass-rates for t] 


other individuals who took the 


1¢ 
examination, 486 individ- 
luis, were unknown. Plaintiffs offered no testimony as 
to the statistical s‘.nifieance of the known figures or of 


the unknown figures. 


Individuals who had been serving as provisional Sup 
HRS’s within the Ageney and who, but for one, had failed 
the examinations. testified to their varied jobs within the 
Agency and to their belief that the examinations were not 
job related. 


Defendants introduced evidence on the structuring of 


the occupational series for positions in the 


new Human 
preparation of 
position of Sup 


Resources Administration. the method of 


the second examination riven for the 


HRS, and the purpose of the examination, to wit, a basic 
knowledge test to permit easy transfer of qualified in- 
dividuals to a variety of jobs on the same level of re- 
sponsibility within the Agency. Defendants’ experts 
evaluated the job analysis prepared by defendant Depart- 
ment of Personnel as consistent with professionally ac- 
cepted standards of test preparation, if not better. 


In rebuttal, plaintiffs’ expert testified to deficiencies in 
the job analysis and stated that he would be unable to 
construct an examination from it. 


The trial ended on January 11, 1974. The parties had 
stipulated to supplement the record for relevant facts in 
the Williams case. In April, 1974, defendants supplied 
the figures for the three examinations challenged in Wul- 
liams. These figures are also set forth in Table IT. Of 
the 106 who took the promotional Sen HRS exam, No. 
1626, 44 passed. Of that mumber 80 Whites passed, for 


l, 
a passing ~»te of SSO; 11 blacks, for a passing rate of 
18%; am. o Hispanics, or 37%. For the open competi- 


tive Sen HRS exam, No. L099, of those identifiable from 
their working at HRA, 101 Whites passed (pass rate of 
65%); 56 blacks (26%), and 8 Hispanies (27%). Of the 
683 persons who took the open competitive, 277 were not 
identifiable. Of that number, 90 passed and 187 failed. 


For the challenged open competitive exam for HRS 
(No. 1097), 606 took the examination, of which 328 were 
identifiable. 59 Whites passed, for a pass rate of 51%; 


55 blacks for a pass rate of 319; 7 Ilispanies for a rate 


of 19%. 


At the same time, defendants introduced statisties for 
the examinations in the series which were not challenged. 
The evidence at trial established that a total of 9 exam- 
inations were given on October 14, 1972 (Tr. 317). One 


examination that plaintiffs did not challenge was the pro- 


motional exam for HRS (No. 1625). Three others were 
for the companion series of exams for Human Resources 
Specialist in the specialty of Manpower Development and 
training (“MDT”). The examinations differed by 10 
questions only at each of the three levels; Sup HRS, Sen 
HRS, and HRS (246; 319), Between levels, there was a 
1) question difference (318), 


Figures for the promotional HRS exam and for the 
Sen HRS (MDT) and HRS (MDT) are set 
Table IT.% 


Ta 


In September, 1974, the District Court indicated that 
it was troubled by arguments raised by 


forth in 


Defendants in 
their brief as to the adequacy of plaintiffs’ case.** By 


figures 
bec: 
was compl 
We assume, for the purposes 


an adverse impact. 


** The statement in the 
defendant's post-trial memorandun is inaccu- 
Insofar as the note implies that an obliga- 


alert the parties or the Court to our legal arguments prior 


our post-trial brief, we did so Contrary to the Court’s State- 


ment, defendants outlined their position in Mr. Bronstein’s Aff- 


davit in Opposition to the motion for a preliminary injunction in 
the Jones case in September, 73, (85a). The 
relied on the affidavit (supra, fn. ] ut apparently had mis- 


placed it by the time of decisi 1, 1 anuary, 1975. The Williams 


Court had 


case was instituted three davs before ‘s when opposi- 


tion to the motion for a preliminary injunction would have been 
futile. The trial in Jones was restricted to the two Sup HRS 
Exams. Had defendants known in January, 1974 that the Dis- 
trict Court would accept in March, 1974 our leral arguments 
in Hill v. Human Resources Administration, 74 Cix 1150, where 
three examinations were at issue. we might well have moved to 
dismiss. Understandably, when Hill was affirmed by this 
cuit, 493 F. 2d 1397 (2d Cir. 1974), we relied on it in our brief, 
dated April 26, 1974. 
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telephone, he asked counsel for their preferences on secur- 
ing expert testimony on the statistical significance of the 
available figures. 


Plaintiffs had apparently proceeded on the theory that 
their statistical case was sound based on the figures them- 
selys. Before trial began, in response to one of defend- 
ants’ interrogatories (169a), plaintiffs stated that they did 
not anticipate calling any expert witness in the course of 
their case in chief and that any expert would be as a 
rebuttal witness to defendants’ testimony on the issue of 
job-relatedness (17la). Even after receiving defendants’ 
post-trial memorandum, they had not moved to reopen 
the record. 


lo the Court’s inquiry, counsel for plaintiffs respo:ded 
that “the issue of the discriminatory impact of the [Sup] 
Open Competitive exam need not be reached” (290a). 
They advanced the legal theory, for the first time in their 
letter, that the promotional exam was the “top” of one 
list, and that if the promotional exam were held invalid, 
“it would ... violate the dictates of both common sense 
and the law to permit appointments to be made from the 
‘bottom’ of the list, ie., the O.C. list” (291a). 


It seemed to defendants that the District Judge was 
on the brink of accepting this * gument. On September 
12, 1974, a letter was sent from chambers stating that 
plaintiffs’ letter of September 10 “properly states the 
considerations which the Court has in mind” and request- 
ing defendants to reply (292a). 

In defendants’ letter of September 24, 1974, they urged 
the Court to adhere to its inclination to secure expert tes- 
timony (293a). In a letter dated October 10, 1974, defend- 
nts protested plaintiffs’ disclaimer of their responsi- 
bility to make out a prima facie case on the open com- 


11 


petitive exams and urged the Court to reject the one 
exam theory “as an alternative to submitting expert opin- 
ion on the significance of the statistics already in evi- 
dence” (295a-297a). 


On October 21, 1974, the Court requested (298a) : 


“That the plaintiffs include in the expert opinion 
evidence which they have been directed to submit, 
an opinion as to whether in the case of examina- 
tions in regard to which the ethnic makeup of a 
group of examinees is partly known, there is evi- 
dence of record from which conclusions as to the 
ethnic makeup and pass-fail rate of the unkonwn 
portion of the group can reasonably be inferred.” 


Fach side submitted affidavits from experts (299a-321a). 
While there was agreement between the experts that no 
statistically accurate conclusions could be drawn about 
the large unknown groups on the open competitive exams, 
the experts disagreed on what conclusions could be drawn. 


The facets are set forth in greater detail in the argu- 
ment portion of this brief. 


Opinion Below 


The District Court found that plaintiffs had estab- 
lished a prima facie case of racial discrimination. While 
no statistical conclusions could be drawn about the un- 
identitied ron-HRA population to indicate adverse impact, 
the Court found it 


“distinctly improbable that minority group mem- 
bers in the non-ILRA (unknown) group would out- 
perform non-I1RA Whites on the same examination 
to the extraordinary degree necessary to bring the 


over-all passing rates for minorities and Whites 


into rough parity. 


The Court believed his conclusion buttressed by a state- 
ment of plaintiffs’ expert that Blacks generally do mort 
poorly tian Whites on examinations consisting of items 
of the type at issue here. Defendants’ statistical expert 
had stated that even hypothesizing an identity between 
the known and unknown groups and the requisite ran- 
domness, the hypothetical pass rate was inconsistent with 

: 


the only known fact, the over-all pass rate, for two ol 


the Open Competitive examinations. This led him to re- 
ject as statistically improbable the hypothesis that the 


disparate impact was the same for both groups. The 
District Court noted that defendants’ expert had not 
stated the degree of improbability, and found, accordingly, 


that it was “] 


} 


ikely” that the disparate impact was the 
same for both groups (335a-336a, n. 7). 


As to the figures for the examinations whieh were un- 
challenged, the District Court found either that the sam 
ples were too small to he valuable or that disparate im- 
pact was shown. Ile found that the over-all pass rates 
for the known HRA group were not significantly altered 
when he added up the figures on the 5 challenged exami- 


nations and compared them to the total on 8 examinations. 


On the issue of job-relatedness, he found that defend- 


ants had failed to meet their heavy burden of showing 


job-relatedness. Ile found numerous deficiencies in the 


efendants, 


ut principally an 


job analysis prepared 1 ( 


insufficiency in the nu 


er OL persons interviewed as to 
the duties and responsibilities of their jobs. He evalu- 


ated the content validity of the exams on the basis of 


4 


opinion testimony and found that they were not content 
valid. 


— 
a) 


After he had decided the case, he addressed himself to 
defendants’ testimony that they were not testing for a 
particular job, but for basie knewledee of the position 
sufficient to permit assignment throughout the Agency. 
He found that they had not established that there was a 
common core of skills which could properly be tested for 
or, if there were, that they had successfully identified 
them. He cited as the most obvious example of their 
failure the fact that on the Sup HRS exam, 20 to 25 
questions related to supervision, whereas only 60% to 
695% of the individuals in these positions actually had 
supervisory responsibility. Further, he noted the LO ques- 
tion difference between the promotional and open com- 
petitive exams for Sup IIRS and said that specific ques 
tions on the functions of the HRA on the former which 
were omitted on the latter in favor of more general ques- 
tions conld not be considered essential] knowledge to all 
those in the title. Finally, if defendants had indeed 
tested for common skills, it was “near y past understand 
ing” why large numbers of provisionals failed, and open 
competitive candidates on two of the exams scored higher 
than those of the promotional candidates, Despite the 
“unique opportunity for a concurrent validation study,” 
defendants had come forward with no evidence to sug 


+ 


sest that plaintiffs were doing an ina equate job. 
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POINT I 


The District Court was clearly erroneous in finding 
that plaintiffs had established a prima facie case of 
racial discrimination. 


In cases alleging unintentional discrimination in public 
employment, statistical evidence can establish that an ex- 
t 


amination had a racially disproportionate impact upon 
minorities. Chane v. Boa of Exa ners, 498 F. 2d 


1167 (2d Cir. 1972), aff’¢ 330 F. Supp. 203 (S.D.N.Y. 1971); 
Vulcan Society v. ¢ Ls ce Commission, 490 F. 2d 


387 (2d Cir. 1973), aff'e 360 F. Supp. 1265 (S.D.N.Y. 
1973): Bridgeport Guardians, Lie. Bridgeport Civil 
Service Commission, 482 F. 2d 1333 (2d Cir. 1973), aff’g 


in part and rev’g in part, 354 F. Supp. 778 (D. Conn. 


1973); Nirkla V.Y. State Dept. of Correctional Serv- 
ices, F’. 2d (2d Cir. 1975), aff'g in part and 
rev’g in part 374 F. Supp. 1561 (S.D.N.Y. 1974). 


However, plaintiffs must prove that there has been a 
“sionificant and substantial discriminatory impact” upon 


the minority applicants who took the examination and 
must demonstrate bv their statistics the existence of “a 


+ 


disparity of sufficient magnitude to amount to a prima 


faci ease o* ‘nvidious di 


supra, 458 F. 2d at 1176. 


facto diserimination’. Chance, 


exacting eriteria have been established in this cireuit 
1 


fo. developing statistical evidence sufficient to support a 


+ 


preliminary injunction. Statistical data of questionable 
probative force due, for example, to samples too small 
to be reliable, have been rejected. In Chance, the Dis- 


trict Court, before it would order a preliminary injune- 
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tion, directed the parties to compile a statistical] survey 

to provide pass-fail statisties. This survey studied fifty 

examinations given over a period of seven vears. 330 

F. Supp. 203, 209 (S.D.N.-Y. 1971 ). In reaching its deei- 

sion the Court relied primarily on nine examinations, 

findine that mean neful conclusions could not be drawn 
from the other forty-one studied. Jd. at 211-213. 

level and promotional examinations of the 

Connecticut, police department were chal- 

in B dgeport Guardians, Inc. v. B dg port Civil 

rervlee Commiss on, 482 F. 2d 133; y iY. 1973). The 


Distriet Court looked at the results of 6 entry level ex- 


aminations given between 1965 and 1970 before deciding 


that the written examinations had a diseriminatory effect. 


Court apparently had exact figures of the numbers 
li took and 

he exams, 354 F. § ipp. 778, 784 (D. Conn. 1973). 
In Guardians Ass'n of Nia © BP, it. Inc. v. Civil Serr 
ice Com’n, 490 F, 2 2d Cir. 1974), an independent 


oreanization was engaged to conduet a survey to deter- 


ites, Blacks, and Puerto Ricans who bot 


mine the racial impact of the two most recent entry level 
examinations. Cf. Carter y. Gallagher, 3 EPD 8205, 
Finding No. 29 at 6668-6669 (D. Minn. 71 mod. on 
other grounds 45 EF. 2d 315 (Sth Cir. 1971) (total results 
of exams given over a 20 year period from 1948 through 
1968 considered): Baker y. Col imbus Municipal Si parate 
School District, 462 ae 1112 ¢! I Cir. 1972) (passing 
rates of Black and White teachers for four tests): West. 
ern Addition Community Organization y. Alioto, 340 F. 
Supp. 1351 (N.D. Calif. 1972) (exaet humbers of white 
and minority applicants who took and passed two writ- 
ten examinations for firemen). 


No less exacting criteria are applied when only one 


examination is at issue. In Vulean Society y. ¢ ivil Serv- 


ice Comnv'n, 490 F. 2d 3887 (2d Cir. 1973), a sight sur 
, 


vey was made of almost 14,000 applieants for an entry 
level firemen’s examination and another of the t yp 5,000 
men on the eligible list who appeared for a physical 
exam. The number of persons identified and the relative 
reliability of the figures led this Court to reject the in 
tervenors’ objections to the statistical case—the City de 
fendants did not challenge on appeal the findings of ra- 
cially disparate impact. 


Similarly, in Wirkland y. N.Y. State Dep't. of Correc- 
tienal Services, supra, complete statistics for the chal 
lengwed promotional exam were known. The Distriet Court 

} 


there correctly recited the applicable standard (874 F. 


Supp. supra at 13869): 


“plaintiffs are required to do no more than demon 


strate that minority candidates as a whole fared 


vy less well than White candidates 


slemificant 


(emphasis supplied) 


In his ruling in this case, the Distriet Judge substan 
tially departed from the strict standards enunciated in 
this circuit as to the proof necessary for establishing a 
prima facie ease. Contrary to the law, he found that a 
prima facie case was established on the basis of incom- 
plete, and hence unreliable statistics. Cf. Nirkland v. 
N.Y. State Dep't of Correctional Services, - en 
(2d. Cir. 1975), slip op. at 5409; Hill v. Human Re- 
sources Administration, No. 74 Civ. 1150 (S.D.N.Y. March 
29, 1974), aff’d 493 F.2d. 1397 (2d. Cir. 1974). 


In the Jones case, two exams, the promotional (No. 
1631) and open competitive (No. 2013) for Sup. HRS 
were challenged. Complete figures were available only 
for the 1388 persons who took the promotional.* <As for 


iss-fail percentages for the 5 examina- 


ficures and the | 


I 


tions at issue are set out in Table I 
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the 994 whe took the open competitive exam, the only 
ones who could be identified were employees of the Hu 
man Resourees Administration. The ethnic identity of 
49% of the candidates (486 994) was unknown. There 
were more unknown passers (183) than the total num 
ber of known passers (172). The ethnic identity of 47% 
of total failures (303/639) was unknown. 


Similarly, in the Williams ease the data is substan- 
tially incomplete. For the promotional (No. 1626) and 
open competitive (No. 1094) examinations for Senior 
HRS, the only persons identified were employees of the 
Human Resources Administration. For the population at 
large who took thre open competitive, the ethnie ile ntity 
and pass-fail rates of 277, or 409% (277/683) was un 
known. The open competitive exam (No. 1097) alone* 
was challenged for the lower level HRS exam, and, again, 


+ 


the ethnie identity and pass rates of 278 were unknown. 
Contrary to the District Court’s conclusion, the figures 
did not show a sufficient disparity among all the minori- 
ties and non-minorities who took the exams to make out 
a prima facie ease: rather, the figures indicated that dis 
parity among HRA employees. 


Putting aside for the moment the question of whether 
the disparate impact on the known HRA population should 
have been enough to establish a prima facie case, there 
was no evidence introduced at trial or in the experts’ 
affidavits that the disparate impact on the known HRA 
population could be carried over to the unknown popu- 
lation. Indeed, no statistical] conclusions whatever could 
be drawn about the large numbers who took the open 
competitive exams at each level, amounting, in deseend- 


rhe promotional HRS exam was not challenged, but the pass- 


fail figures resulting from it were introduced into evidence. 


ing order, to 49%, +() and 46 of the total exam tak 
ing populations 

The only case in this cireuit which has dealt in any 
way with a broad-based att ries of examina 
tions where ethnie and corresponding pass-fail data were 
substantially unknown for la portions of the separate 


exam-taking populations was //i// Human Resource 


Administration, No. 74 Civ. 1150 (S.D.N.Y. Mareh 29, 
1974), aff'd 493 F.2d 1397 (2d. Cir, 1974). 


In Will. three in another series of aminations given 
for positions in the Huma Resou Administration 
were challenged. While all wer Opel ¢ mmpetitivi exams, 


the lowest level examination was not written, but a “train 


ing and experience” exan hie andidates are given 
scores based on those factors (s¢ 180-181). The ethnic 
identity of only 9% of the candidates was known. The 


second level, however, was a written test. 704 persons 
identity of 440% of the passers was known; Whites passed 
at the rate of 72%; Blacks at t rate of 45%; His- 
panics, at the rate of 50%. For the third examination, 
also written, 404 took t 
ethnic identity of 28% of the passers was known. Whites 


took the test: 215 passed and 487 failed. The ethnic 


e examination; 266 passed. The 
passed at a rate of 90% ; Blacks at the rate of 61%; 
Hispanics, at the rate of Jot. The District Court found, 
in part, that thes fieures were too unreliable to estab- 
lish a likelihood of success on the merits, Le, a prima 
facie ease. The denial of the preliminary injunction was 
affirmed without opinion by this Circuit. 493 F. 2d 1397 
(2d Cir. 1974). 


1 
{ 


In the instant case, the District Court distinguishes 
Hill in a footnote, saying that plaintiffs here had pre 


} 


sented data as to a considerably higher proportion of 


candidat fowever, t] 44 % of passers known for 
the second level examination in J/ill is 
different from the 4 


ot known passers for 


2013, for Sup HRS. In gross numbers alone, these wer 


more unidentified candidates here than in //ill. To dis- 
tinguish the cases on the higher number of known ean 
didates avoids the issue. 


The District Court went on to say that “substantial 
disparate impact was not shown, even as to the avail- 
able samples, in all three examinations in Hill.” This 
was «due, in part, to the accident ¢ lnintiffs’ timing of 
their challenge. By the time suit w instituted in Hill, 
defendants had already begun to interview eandidates 
from the eligible list. For the written exam where 44% 
of the passers was known, and Whites passer. at a 72% 
rate compared to 45% and 50% for Blacks and His- 
panies respectively, of the 64 persons who app ared for 
interview when called from a certified list containing 120 
names, 43 were minorities and 21 White (Appellants’ 
Appendix in //il, p. 59a). In this ease, defendants were 
not permitted to call the list for interview because of 
the issuance of the preliminary injunction. Had those 
on the list been called, the overall impact of these ex- 
aminations could, of course, have been more accurately 
gauged. By the same token, even without allowine the 
listed candidates to be called, the actual impact of these 
examinations on White and minority candidates could have 
fairly been ascertained by survey methods, an under- 
taking which plaintiffs never attempted, and which, we 
contend, should have been required by the court below. 


In Kirkland v. N.Y. State De pt. of Correctional Serv- 
ices, - F.2d —— (2d Cir. 1975), slip op. at 5409, 
this Court made reference to the “incomplete, and there- 
fore unreliable” data submitted with regard to previous 
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examinations. Tr Dougiys v. Hampton, —— F.2d —— (D.C. 
Cir, 1975) a tistical survey was made of approxi- 
mately 50,0" love graduates of the 153,000 who cus- 
tomarily to ie FSEE test: the “sample from which 


the statistic: was derived was random, in the statistical 
sense. and larce enough to be significant.” There is no 
areument here that the identification p: cedures were de- 
termined on a random basis. Cf. Jill v. I7uman Re- 


sources Administration, supra, p. 9. 


An affidavit of plaintiffs’ expert, Dr. Richard Barrett, 
was submitted on the statistical aspeet of the case. He 
used the Chi Square test to show that there was a dis- 
parate impact upon minor ties who work for HRA and 
who took either the promotional or open competitive 


¢ 


exams (303a).* 


The major issue was, the Court noted, sti:] unresolved: 
“Whether the data for the entire group of condidates on 
Nos. 2013, 1097, and 1999 would show the sume results 
as Barrett calculated on the basis of the known eandi- 
dates on those examinations” (333a). There was no 
way of knowing, as Barrett admitted, “strictly speaking”. 
Defendants’ expert, Professor Grammas, agreed (310a): 


“Tt is my opinion that neither racial break-down 
nor pass-fail can be inferred about the unidenti- 
fied test-takers from the known group and _ that 
there is no statistical basis for postulating a ra- 
cially disparate impact on the results available for 
the known group.” 


* Tt should be noted that Dr. Barrett did not use the Chi Square 


test for Hispanics ‘ately from Blacks, although the sar:ple 
size was, except for exam No. 1625, undoubted sufficient. Cf. 
Chauel, supra, 330 F. Supp. at 213. The pass rates for Blacks 
and Hispanics did nt show the same disparity on all the challen- 


ged exams (316a). 
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In short, no statistical conclusions would he drawn 
about 49%, 40%, and 46%, of the separate exam taking 
population for the three exams. 


The District Judge solved his dilemma by substituting 
a personal standard of probability for a statistical stand- 
ard of probability. The opinion quotes from Dr. Barrett 


(333a): 


“Strictly speaking such a determination can be made 
only if there is reason to believe that those whose 
identity is not known are a random sample of the 
total group. There is, of course, no way to make 
this determination. However, the size of the Chi 
Square statistics reported above ... is so great 
that those whose race or ethnicity is unknown would 
have to differ in an unrealistically laree degree 
from those whose identity is known to lead to the 
conclusion that the tests are free from adverse im 
pact.” 


Dr. Barrett went on to say (302a): 


“In the absence of compelling evidence that those 
of unkown identity are from a totally different 
group than those whose identity is known the re- 
sults of 

well,” 


the above analysis must apply to them as 


While there is no statistical basis for applying the re- 
sults of the known group to the unknown group, Dr. Bar- 
rett is prepared to assume they apply unless defendants 
prove otherwise. The District Court adopted his rea- 
soning. 


Concluding that Barrett’s affidavit accorded with com- 
mon sense, the District Court found it (384a): 


99 


rare 


“distinetly improbable that minority group mem- 
bers in the non-IHRA (unknown) group would 
outperform non-HRA whites on the same exam- 
ination to the extracrdinary degree necessary to 
bring the overall passing rates for minorities and 
whites into rough parity.” 


This type of analysis undereuts the purpose of the 
statistical case. The District Court’s expectations as to 
the performance of minority applicants have no place in 
determining whether plaintiffs have established a prima 
facie ease of racial discrimination hased on hard statis- 
tical data. See Chance Vv. Board of Examiners, supra, 
There is no statistical probability, one way or the other, 
as to the performance of the unknown candidates or even 
as to their identity. There are not two different stand- 
ards of probability, one when a statistically sound ran- 
dom sample is available; another when it is not. In eases 

ae of this nature the courts deal only with statistical prob- 
abilities: “statistical evidence by its very nature deals 
with probabilities rather than certainties.” Vulean y. Civil 
Service Commission, 360 F. Supp. at 1270. 


In imposing his own personal standard of probability, 
the District Court exceeded his “limited office of .. . find- 
ing that black and hispanic candidates did significantly 
worse in the examination than others”. Vulcan v. Civil 
Service Commission, 490 F. 2d supra, at 390. 


The District Court believed that his conclusion as to 
the probability of minority performance in the unknown 
population was buttressed by Dr. Barrett’s observation, 
contained in his post-trial affidavit, that exam Nos. 2013, 
1097 and 1099 are “made up of items of the type on which 
Blacks and Hispanics generally do more poorly than 


whites.” This statement was not subject to cross-exam- 


ination. The statement may be Dr. Barrett’s personal 
opinion.* At best, its meaning is unclear. There 


is 
“overwhelming evidence that. 


on the average, black peo- 
ple and other disadvantaged groups perform 


substantiall 
less well than Whites on reneralized ‘ 


intelligence’ or ‘ap 
titide’ tests.” Cooper and Sobol, Seniority and Testing 
Under Fair Employment Laws: A General Approach to 
Objective Criteria of Hiring and Promoting 82 Hary. L. 
Rev. 1598, 1639 (1969). See also Griggs v. Duke Pou 
Co., 401 U.S. 424, 430 (1971), dealing 
aptitude tests. The 


: 
with standardized 
tests at issue here were not 


tests. Rather, they were tests geared to the content of 


such 
the position. 


Testimony of this nature has not. bed n 


relied on, if 
mentioned at all. 


in prior cases dealing with di iscrimina- 
tion in publie employment, where written tests 
volved. See, e.g., Chance v. Board of 
Vulcan v. Civil Servier Commi 


were in- 
Examiners, supra; 
Ss70on, supra; Kirkland v 
N.Y. State Dept. of Correctional S: rvices, supra. Indeed, 
it has ominous implications for civil service jurisdic 
uch as New York where most examinations are ecom- 
petitive written examinations. See Kirkland v. New York 
State Dept. of Corre ctional Service s; K’.2d (2d 
Cir. 1972), slip op. at 5411, 5412. 


tions 


Defendants below had cited Gonzalez 
York, 4 E.P.D. 9 7867 (S.D.N.Y. 


tion that “numbers comprise a 


v. City of New 
1972), for the proposi- 
eritical clement of what 
the case purports to be about” and that the 
there were “neither fully revealed nor d 


decisive in their 
incomplete state at the moment.” The Court distinguished 


numbers 


a 


* Dr. Lopez, plaintiffs’ other expert, testified 
ledge test a pen and pencil examination should be given if the 
applicants know how to read and write (Tr. 684). 


that for a huow- 
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Gonzalez on the ground that the only figures known there 
were of the 5 plaintiffs. If a Court is prepared to rely 
on statements as to the general performance of minorities 
on “items of (this) type” the sufficiency of the prima 
facie case becomes a matter of the Court’s imp-sssion, 
and not a matter of statistical proof. Argnuabiy, on the 
same figures, Hill could be decided differently. 


The combination here of two erroneous elements, the 
District Judge’s personal standard of probability plus a 
statement as to general performance of minorities on 
“items of this type”, does not make up for the deficien- 


cies of proof in plaintiffs’ statistical case. 


The Court below also betrayed a serious misunderstand- 
ing of the opinion of defendants’ statistical expert in stat- 
ing that his opinion “is not inconsistent with our conelu- 
sions as to 2013, 1099, and 1097”. Professor Grammas 
flatly stated that there was no basis for drawing any 
statistical conclusions as to the unknown population, for 
the obvious reason that nothing was known about. that 


group. He was not invited to give his personal views 
as to the likely performance of unidentified minorities. 
In the final analysis, only the Court could decide what 
was or what was not inconsistent with its belief as to 


a “realistic likelihood” of performance by minorities since 
his personal belief was not susceptible of statistical proof. 


Similarly, the District Court se..us to have missed the 


point of Dr. Grammas’ postulation of a set of assump- 
tions. Nothing is known about the non-HRA population 
except the over-all pass rate. Professor Grammas was 


willing to test the hypothesis that the groups were iden- 
tical* against the only known fact, the over-all pass rates. 


* Dr. Barrett simply assumed this to be true in the absence of 
contrary “compelling evidence”. 


or 


“0 


To do so, he had to assume that the non-HRA popula- 
tion broke down identically to the HRA population as 
to race and pass-rates and that the non-HRA popula- 
tion was a random sample of the total population—other- 
Wise, even hypothetieally, he would be measuring apples 
and oranges. Professor Grammas found that even mak- 
ing the assumptions, the hypothetical pass rate signifi- 
cantly differed from the observed pass rate for two out 
of the three open competitive exams at the 95% level of 
confidence. This meant that at least on of the three 
assumptions was invalid, although there was no way of 
knowing which one. Professor Grammas, as a statisti- 
cian, would reject the working hypothesis that the dis. 
parate impact was the same for the unknown groups as 
for the known groups on two of the exams (314a). 


If anything, this statistical analysis should have given 
the District Court pause. Instead, it becomes fodder for 
its personal impression of the statisties. Thus, the Dis- 
trict Court faults Dr. Grammas for not indicating the 
“degree” of probability “other than to indieate that it 


is something less than 95% probable”. Both statistically 


A 


and legally the 95% level of confidence is the standard 


level at which figures are considered Statistically signifi 
eant.* <Albermarle Pape Y Co. % Moody, ——~ U.S. 
(1975), 45 ZL Ed 2d 280, 304. Notably, the same eriti- 


Pee ae 
* The District Judge goes on to point out that none of the 
three a sumptions Professor Grammas made had to be true as a 
matter of fact. This is, of course, true if > had sufficient in- 
formation about the non-HRA group from which one could draw 
statistically reliable conclusions. The point is that there is no in- 
formation about the non-HRA group except the over-all pass rate, 
and Professor Grammas merely tested by accepted statistical 
the hypothesis that the pass rates were the s; in both 
against that known fact. 
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cism is not applied to Dr. Barnett, who is not required 


I 
| 


istically large degree” is by 


to indicate what the “unrea ; 
which the pass-fail rate of minorities in the unknown 
eroup would have to differ from the rates in the known 
group. The Court’s own reference to an “extraordinary 
degree” is a matter of its own impression. 


The startling conclusion the Court reaches is that, since 
defendants did not prove how improbable “improbable” 
was the Court was free to say that it was probable: “al- 
though it is not statistically certain that the non-HRA 
groups showed the same disparate impact as the HRA 
croups, it is indeed likely that they would (336a).” 


We submit that the Court’s analysis is incorrect. Plain- 
tiffs had the burden on their case in chief to prove, by 
reliable and significant statistical evidence, that the ex- 
aminations had a disparate impact on minorities as a 
whole. Chance v. Board of Examiners, supra; Kirkland, 
supra, —— F. Supp. at ——. When they failed to make 
he showing, the Court was not free to substitute its 
own personal standard of probability nor to shift the bur- 
den to defendants to disprove incomplete statistics. The 
only probabilities which the Courts will recognize are 
statistical probabilitic s, Vulcan Society v. Civil Service 
Commission, supra, 360 F. Supp. at 1270. 


Defendants introduced figures on two open competitive 
exams for the sister series of the Manpower Development 
and Training (“MDT”) specialty, Exam. No. 1094 for 
Sen HRS (MDT) and Exam No. 1095 for HRS (MDT)* 
as well as figures for the unchallenged HRS promotional 
exam No. 1625, set forth in Table IT to the brief. They 


* The figures for the Sup HRS (MDT) exam were not intro- 
see fn. p. 9, supra. 


were introduced for two purposes: (1) to point up the 


weaknesses in plaintiffs’ prima facie case based on sub- 


stantially incomplete, but impressive looking, statistics; 
and (2) to bring to the Court’s attention the consequences 
of challenging part of a series of substantially identical 


exams. 


The statistics for the MDT exams and the promotional 
HRS exam did not show, on their face, disparity (Nos. 
1094 and 1625) or, disparity to the same degree (No. 
1095).* These examinations were 87.5% identical at each 
level to the ones ehallenged. Furthermore, the promo- 
tional HRS exam was conspicuous by its absence among 
the challenged exams, since the promotional exams for 
Senior and Sup HRS were challenged and were indeed 
the only examinations on which complete figures were 
available. 


In short, not only were plaintiffs’ statisties incomplete 
with respect to the challenged examinations, but also 
plaintiffs’ selective attack on only certain of the exam- 
inations in this series is misleading in not showing the 
entire, overall impact of the series of examinations. Un- 
der these circumstances, we submit, it was clearly er- 
roneous for the District Court to hold that the plain- 
tiffs had made out a prima facie case for setting aside 
the challenged examinations. As we point out in the 
following points, in fact these examinations were job re- 
lated, but before even getting to consideration of these 
points, we would urge that this Court should most care- 


* It should be remembered that at the time of post-trial briefs, 


a | 


10 expert testimony had been introduced. The failure to offer 
evidence as to the statistical significance of the differential i 


rates for three exams was grounds for reversal of a prelimi 


injunction in Commonwealth of Pennsylvania v. O'Neill, 
FE: 2d 1029 (3d ‘Cir. 1973). 
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fully consider—and reject the approach adopted here by 
the District Court in deciding that these plaintiffs had 
made out a prima facie case. Approval of that approach 
can only have the most disastrous consequences Tor civil 
service jurisdictions generally, and for this ageney in 
particular. In addition, it may well work unfair hara 


ship on successful candidates on such examinations. 


Should plaintiffs prevail, and the five challenged exams 
be struck down as constitutionally invalid, there would be 
a substantial group enjoying the benefits of permanent 
civil service appointment from the four unchallenged 
tests, although they would have succeeded on the basis 
of exams 87.5% identical to the challenged exams. This 
could only | iten the resentment and divisiveness among 
the candidates which Judge Weinfeld sought to avoid in 
Vulcan, supra, 360 F. Supp. at 1278. The result would 
be particularly harsh for the open competitive HRS exam. 
The promotional exam, on which Blacks outperformed 
Whites, was given for the very same position for which 
the open competitive exam was given, yet appointments 
were based on that examination. 


In addition, striking down these examinations may very 
vell be unfair to the minority group members who took 
and passed them. Here a total of 197 minority group 
candidates out of a total of 540 (340a) or 36%, passed 
these examinations; these individuals can now not be ap- 
pointed to the permanent civil service positions to which 
they may be entitled. 


These hardships are in addition, of course, to the hard- 
ship imposed on the public by setting aside these ex- 
aminations if in fact they were proper examinations. Un- 
der all of these circumstances we submit that it was 
error for the District Court to hold that plaintiffs had 
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made out a grima facie case with their partial and se- 
lective statistics. 


As has generally been required in cases of this na- 
ture, plaintiffs should have been required to make a prop 
er statistical case—based on a study of the universe of 
candidates who took these examinations or on a proper 
survey of such candidates. This plaintiffs never did, and 
this should be considered a fatal defect in their proof. 
On this ground alone, their complaints should %e dis- 


missed. 


(2) 


It is our position that even as to the challenged pro- 
motional examinations, where, concededly, disparate re 
sults were shown between whites and minorities were 
shown, it should be held that plaintiffs failed to make 
out a proper statistical prima facie case. We take the 
position for the reason that these examinations were 
in fact part of a related series of examinations and the 
number of candidates on each of the promotional ex- 
aminations was relatively small. Under these circum- 
stances, where the evidence was available to plaintiffs 
to put in a proper statistical showing on the entire series 
of examinations, but they chose not to do so, they should 
not be allowed to succeed in a selective challenge to only 
a part of the series. If plaintiffs had tried their case 
the way they should have been required to try it, i.e., by 
showing the results of a proper survey of the entire 
series, the complete picture as to the effect of these ex- 
aminations would have been known. Plaintiffs should not 
be allowed to succeed, even partially, as a result of their 
own default in coming forward with proper proof. 


As it was, 35% of total passers (15/43) for the Sup 
HRS promotional list and 32% (14/44) for the Sen HRS 
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promotional list, were minorities. This is a far ery from 
Vulcan, where only 4.5% of those who passed the qualify- 


ing medical and physical examinations as well as the writ 


ten examinations were minorities (360 F. Supp. supra at 
1269), and minorities COMPTise | O1LL\ o of the ire 
Department (Jbid, n. 11). 


The District Court in its opinion had already gone 
through the reasoning discussed above to find that plain 
tiffs had made out their prima facie case. Tt therefore 
took the approach that the figures defendants had in- 
troduced, if defendants were to b uccessful in their 
defense, had to rebut plaintiffs’ established prima faci 
ease. It found that they did not, and disposed of our 


equitable arguments “more easily”. 


The District Court found that the White sample for 
KMxams No. 1094* and 1625**, was 1 small to be valu- 
able and that a change in figures by merely two would 
change the results. As to Exam No. 1095, “only 17 whites 
took the exam” but in any event whites passed at a sig 


Ss, again, n t lb t Court conclusion 
lite mple o 15 | 1) small to be 
In Cha , as the District Court itself pointed out, 
Mansfield relied on figures from the nine examinations 


F. Supp. at 


The smallness of the sample may be a factor in a Court’s not 


relying on it in assessing the statistical case. However, if plain- 
tiffs had made out their case on the open competitive exam alone, 
and the promotional figures for an exam 87.5% identical were 
statistically insignificant, there would be all the more re 1 hv 
the fate of the promotional exam should b ined to that the 
open competitive Instead, it “easily d sed” of our argu 


ment by holding that plaintiffs are not required to challenge more 
than one examination, citing Vu/can where a comparable situation 


did not exist. 
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nificantly higher rate than minorities or blacks. To demon- 
strate the “shaky factual basis for defendants’ argument”, 
he aggregated the results first on five exams and then 
on eight exams to show that the “overall pass rates are 
not significantly altered.” 


In aggregating the results, the District Court fell back 
on the statistics available for the known HRA _ popula- 
tion and ignored our equitable arguments. While the fig- 
ures in the aggregate may have appeared impressive to 
the Court, they do not advance the understanding of the 
issue; one would expect similar pass rates for the HRA 
employees who presented themselves for the MDT exams 
“because it is the same group, HRA employees, and al- 


most the same exam. It provides us with no information 
about the non-HRA group” (318a). 


The point of introducing the statistics from the three 
MDT exams and the promotional HRS exam was not to 
prove that no disparate impact was discernible on those 
exams, and therefore, that plaintiffs had not made out 
their prima facie case. The incomplete statistics on the 
MDT exams could hardly be introduced by defendants 
as statistically significant when we disputed the challenged 
exams on precisely those grounds. Our argument was 
that their varying results should have confirmed the prop- 
er legal conclusion that the incomplete statistics on the 
chailenged exams were insufficient to make out a prima 
facie case and, considering the impact on the public in- 
terest, that equitable relief should be denied. 


The District Court was clearly erroneous, as a matter 
of law, in holding that incomplete, and hence unreliable, 
statistics were sufficient to establish plaintiffs’ prima facie 
ease; and, in failing to give proper consideration to the 
effect on the public interest if plaintiffs were to pre- 
vail and substantially identical exams were allowed to 
stand, abused his diseretion in granting injunctive re- 
lief. The decision must be reversed. 


POINT II 


Defendants’ method of preparation of the examina- 
tions was consistent with professionally acceptable 
standards of test preparation. The District Court 
was clearly erroneous in finding otherwise. 


The District Court correctly recited the “ground rules” 
in cases of this nature: once plaintiifs have established 
a prima facie case of racial discrimination, the burden 
shifts to defendants to prove that the examinations are 
job-related. Kirkland y. N.Y. State Dep't of Correctional 
Services, —— F. 2d (24 Cir. 1975), aff’e in part 
and rev’g in part, 374 F. Supp. 1861 (S.D.N.Y. 1974); 
Vulcan Society v. Civil Service Commission, 490 F.2d 387 
(2d C'r. 1973), aff’g 360 F. Supp. 1265 (S.D.N-Y. 1973). 
Notwithstanding the lack of any job analysis in Vulcan, 
Judge Weinfeld evaluated the examination itself, and, 
based on the testimony at trial, concluded that defend 
ants had not sustained their heavy burden of proving 
that their test was “reasonably constructed to measure 
what it purports to measure.” 360 F. Supp. at 1272 
quoting Chance, 458 F.2d at 1174. 


This Court approved the approach taken by Judge 
Weinfeld (490 F.2d at 395): 


“As we read his opinio:, the judge developed a sort 
of sliding scale for evaluating the examination, 
wherein the poorer the quality of the test prepara 
tion, the greater must be the showing that the ex- 
amination was properly job-related, and vice versa. 
This was the point he made in saying that a show- 
ing of poor preparation of an examination entails 
the need of ‘the most convincing testimony as to 


job-relatedness.’ The judge’s approach makes ex- 


cellent sense to us. Jf an examination has been 
badl.- prepared, the chance that it will turn out 


to be job-related is small. Per contra, careful prep- 
aration gives ground for an inference, rebuttable 
to be sure, that success has been achieved. A prin 
ciple of this sort is useful in lessening the burden 
of judicial examination-reading and the risk that 
a court will fall into error in umpiring a battle 
of experts who speak a language it does not fully 
understand. See Chance, supra, 458 I’. 2d at 1173.” 


While paying lip service to the applicable law, the Dis- 
trict Judge in this ease turned the legal standard on its 
head, The Distriet Judge’s thinking was so dominated 
by the deficie he found in job analyses which were 
admittedly con. .entiously prepared,* that he f. ed to 
evaluate the examinations themselves in any detail, if at 
all. In effect, defendants were graded on their job analy- 
sis. Indeed, the District Judge had already decided the 
case against defendants on the basis of the job analyses 
before he gave serious consideration to the purposes for 
which detendants said they were testing. Ironically, de- 
fendants had fared better in Vulcan where there was no 
job analysis at all. If the decision here is read in the 
order in which it is written, the reader ean only con- 
clude that the District Court did not have clearly in mind 
what the position of Human Resources specialist entails 
or what defendants tried to accomplish by the examina- 


tions. 


It becomes necessary, then, for defendants to begin 
where the District Court left off: to deseribe the duties 


* The sincerity of our efforts was largely responsible for the 


denial of plaintiffs’ request for attorneys’ fees (365a-370a). 


and responsibilities of the Muman Resources Specialists 


noses for which they were ested. 


and the pu 


The Human Resources Administration was created in 
1966 to consolidate under one roof a variety of welfare, 
social services and man-power programs (349a, n. 14). The 
mission of the agency was “to combat poverty and human 
misery” (Defs. Exh. J, “HRA Basic Facts 1972”, p. 7). 


Ambitious as was the concept of ihe agency, its staff- 
ing was even more innovative. One of the considerations 
in establishing the agency was an attempt to bring minori- 
ties into the operational structure of City government, 
rather than only serving them as part of the Agency’s 
client population. The Agency went out of its way to 
served provisionally (344). 


{ ) 
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i 


recruit minorities who at first 


There then began the lengthy classification process to 


set up the civil service occupational series, For each 
proposed title, e.g., Human Resources Special'st, the fune- 
tions of the position, the qualifications necess ry, and the 
selection criteria were developed jointly by the New York 
City Department of Personnel and the Human Resources 
Administration (122a-124a). 


In 1968, because of the large numbers of provisionals 
serving, the Civil Service Commission ordered that ex- 
aminations be held (Tr. 343). As soon as the Notices of 
Examination were published, the City was sued. Matter 
of Lillard v. Ginsberg (Sup. Ct. Ne) ee Oe. Biedoal ss 


7/12/68, p. 10, ¢. 5. Petitioners there argued that they 


were federal, not City emplovees, and henee not subject 
to the requirements of the City’s Civil Service system, 
and, further, that the administration of examinations was 
antithetical to the broad purposes of the Economie Op 
portunity Act requiring the maximum feasible participa- 
tion of the poor. In dismissing the petition, Justice Mur- 


phy used language which was prophetic of the law to 
come in the area of discrimination in public employment: 


“The assumption [is] that the ‘poor’ will be un 
able to qualify for and achieve satisfactory scores 
in Civil Service tests. Thus, the Civil Service, or 
merit system once urged by the then avant-garde 
social philosophers as necessary to insure that those 
then diseriminated against would be able to obtain 
government employment based, solely, upon merit, 
is now urged to be a roadblock to the upward mo- 
bility of the present poor.” 


The examination for Sup HRS in 1968 was so geared 
to protecting incumbent provisionals that credit was given 
for working in the Agency (Tr. 345-346); a mandatory 
experience requirement was ‘at least one year... in an 
agency whose primary function is the administration of 
anti-poverty programs such as community action and de- 
velopment manpower and skills training for the disad- 
vantaged, community relations, ete.” (Defs. Exh. C). 


The list established as the result of the 1968 training 
and experience exam for Sup HRS was broken down into 
three “specialities’* or “parentheticals”, namely (1) Man- 
agement, (2) Community Programs, and (3) Manpower 
Development and Training ({Ibid.). The term “speciali- 


« At the outset of the trial, all the indications were that plain- 


tiffs’ crallenge to jov-relatedness was based on the failure t 


T 
I 
) 


give 
examiuations in the “specialties” as distinct from the “generic” 
title (54a-57a; Tr. 34-37, 58). In view of the adverse testimony 
of their exvert on this point (Tr. 656-657) and the fact that the 
lists from ihe wo lower titles were never broken down into spe- 
cialist (Tr. 39; 64). Plaintiffs abandoned their position. In any 
event the Court did not base its decision on the difference between 
them. 


9° 
vb 


ties” was a short-hand reference to the individual depart- 


Q~ 


ments to which eligibles would be assigned (Tr. 350). 


There was an additional reason for breaking the list 
down into specialties: it created small lists and, in ef- 
fect, assured that incumbents would be appointed (Tr. 
346). At the same time, it permitted the agency to ap- 
point provisionals to the “generic” title of Sup HRS (Tr. 
348-349) without running afoul of the Civil Service Law 
since there was no outstanding eligible list for the generic 
position, only for the specialties. 


During this time, the Human Resources Administra- 
tion was rapidly expanding (Tr. 348; 350). Within six 
months of the establishment of the 1968 lists, the lines 


of demarcation between the specialties was obscured (Tr. 


348). There was a commingling of the specialties and 

generic positions throughout the Agency (Tr. 348; 359), , 
People were transferred as the needs of the Agency dic- 

tated (Tr. 349). 


In 1971, the Civil Service Commission ordered new ex- 
aminations for the “generic” position of Sup HRS, a posi- g 


tion in which approximately 100 provisionals were serv- 
ing (Tr. 226; 248; ef. 374). 

Leonard Rosenberg of the Department of Personnel 
was assigned to the Bureau of Examinations and in 1970 
was made responsible for the Human Resources occu- 


* The number of persons serying 


* provisionally becomes impor- 
tant in assessing the sufficiency of the sample interviewed by L.eo- 


nard Rosenberg. Defen 


lants’ Exhibit L ws that approximately 
180 provisionals were serving in December, 1973. In light of the 


evidence, it is inconceiy ible that the number of provisionals in 


this rapidly expanding agency remained constant for over two 
years, between October, 1971, when the b analysis was pre- 


pared, through December, 1973, the eve of trial. Cf. 374-376 
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pational series (Tr. 172). He worked closely with the 
then Director of HRA Personnel, Harold Basen, and the 
Assistant Director of HRA Personnel, Howard Matelli 
(124a). For the year and a half before the examina- 
tions in issue were ordered, he had contact with Agency 
officials on the average of two or three times a week 
(Tr. 175); 50% of his time was spent on HRA mat- 
ters (Tr. 221). He became involved “with the title strue- 
ture itself” (Tr. 219; 221). He brought to his assign- 
ment long-time experience in classification work, in which 
a job description is developed based on the work per- 
formed (Tr. 172-178; 366). 


Rosenberg in all probability took the examination in 
the Personnel Examiner series prepared by plaintiffs’ ex- 
pert, Dr. Felix Lopez (Tr. 651). Lopez testified that per- 
sons who took the test, like Mr. Rosenberg, would have 
to know the purposes of * job analysis and be familiar 
with the various ways of conducting it (Tr. 655). 


Rosenberg contacted HRA in his preparation for the 
challenged series of exams and asked them to arrange 
a serics of visits. His restriction was that he eould in- 
terview only those incumbents holding the permanent title 
of Sup HRS or any individual holding a higher title (Tr. 
193).* He saw those persons designated by the Agency 
in interviews lasting a week (195ael1). 

Of approximately seven permanent Sup HRS’s, Rosen- 
berg interviewed four, and four additional persons in 
higher titles, and observed the work of others on the job 


undue advantage that might flow to the person who is 


tor a description of the duties and responsibilities of the p 
to be tested for on the examination. It serves a salutary purpose 


in the well-established City agencies. 


At t 
had a 
Court 
(349a) 
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persons interviewed was Toward 
Director of Personnel. 


lie time he did his interviewing, Rosenberg already 
2ood erasp of what S \] RS’ lid. The District 


relied on Rosenbere’ job ¢ | point out 


“Employees holding the generie title Sup HRS may 
do jobs ranging from payroll and purchasing of 
supplies to public relations or program planning 


(Defendants’ Exhibit IF) in twenty different kinds 
of programs (Tr. 581). Moreover, although there 
are a number of small clusters of Sup HRS’s wh 
do approximately the same kind of work, there sre 
no large-sub-groups capable of casy categorization 


a\ 


}. 


It was clearly impossible ver 100 individuals 


doing’ 
Jobs, 1 
be tes 
of the 
they 1 


different jobs on the specifies of their individual 
ior was that the Citw’s intent. fndividuals were to 
ted for sufficient basic knowled * the funetions 
Ageney and sufficient skills to ce ‘ work which 


night be reasonably called upon to perform as a 


Supervising Human Resources Specialist (199; 241; 252). 


Mr. 


1 
Rosenberg t 


“We are not giving an examination for each and 
every position that holds the title of Supervising 
Human Resourees Specialist. We are givine an 


ar) 


examination for the title whiel cnecompfisses a va- 


riety of people in various types of activities with 


ly all being considered similarly alike to warrant 
the same title and the same pay. 


We were giving an examination on the basis of all 
the knowledge we had for the total position which 
at that point in time, comprised over a hun- 
dred individuals, and might ¢row more or might 
shrink. We didn’t know what the future would 
bring. 
And we devised a plan that tests for the ability of 
the individual so é pe ee they vet transferred 
... [they might] perform satisfactorily regardless 
of the exact assignment that the individual mieht 


receive at any one point in time.” 


And, as plaintiff Jones testified (Tr. 54): 


“At the time T took the examination I had no idea 


[ would ever be working in labor relations. At the 


time I took examination, IT was working in per- 
sonnel.” 


The job analysis was introduced as Defendants’ Ixhibit 
F (195ael-3). There were eight areas in which Sup HRS’s 
should be proficient, running the ramut trom knowledge 
of the functions of HRA and its constituent agencies to 
language skills and chart interpretation (Ibid. F). 


The District Court found that a sample of four em- 
ployees in the Sup HRS title was critically insufficient for 
an understanding of the duties and responsibilities of the 
position. Since this factor figured so prominently in the 
Court’s decision—indeed, his entire Jecision hinged on 
it—we discuss it at leneth. 


A professionally aeceptabl impling is anywhere from 
10-25% (Tr. 499; 582). That is, if the City had engaged 
the services of an independent psychologist, such as Dr. 
Lopez, to prepare an examination for the position of Sup 


+0) 


HRS, about which he knew nothing, a sample of 10% 
might be sufficient (Tr. 582). As noted above, in 1971, 
when the job analysis was done, there were approximate- 
ly 100 provisionals (Tr. 243). A total stranger to the 
City and to the Agency could have interviewed ten per 
sons and that number would not be inconsistent with pro- 


fessionally acceptable standards. 


The exact number of persons interviewed by an out- 
side expert in testing would of course be relevant in de- 
termining whether the interviewer had obtained a good 
cross-section of the duties of the posit n (672). Where 


there is a Department of Personnel in existence, a fair 


sampling depends on the individnal who is assigned the 
examination, the degree to which the individual is aware 
of the operations of the ageney, the types of contacts 


and the levels of contacts the individual has with that 

agency on a day-to-day basis (Tr. 229). The exact num- 

ber, four in the Court’s view, cannot fairly be deter- 

minative iere, where Rosenberg was intimately familar 

with the contours of the position: he had been in con- 

tact with agency officials for a year and a half on the 
3 


) 


average of 2-3 times a week (Tr. 175). 


The District Court dismisses Rosenherg’s prior experi- 
ence in HRA matters on the ground that his “private 
knowledge” could not have been of benefit to the per- 
son constructing the exam unless such knowledge was 
committed to writing in the job analysis. However, the 
purpose of a job analysis is to become familiar enough 
with the position so that a determination can be made 


ny 


as to which knowledge or skills ean and should be tested 
for. This is an inferential process (Tr. 525). Rosen- 
berg had designated those areas in the test plan (195af1). 


Once the areas were designated, the test constructor tales 
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over (285). It is not necessary to tell the test constructor 
every detail about the job (ef. 652). 


Helene Willingham, who prepared the examinations, 
was herself acquainted with the position (259; 262). In 
any event, in this age of specialization, it is standard 
practice to divide the functions of job analysis and item 
preparation (Tr. 406). 


As it is, there is ample corroboration that Rosenberg 
had a very thorough knowledge of the duties of a Sup 
HRS. Plaintiffs deseribe their duties in essentially the 
same way that Rosenberg did (47a-49a; 60a-62a; 65a- 
66a; 197a-199a). If Rosenberg had interviewed all the 
plaintiffs’ witnesses who testified, it is hard to see that 
his knowledge of the position would have been appre- 
ciably changed. Furthermore, the “tasks observed” were 
carried over into the Notice of F-xamination as “examples 
of typical tasks.” If the “typical tasks” were not de- 
scriptive of the jobs they were doing, it was open to 
plaintiffs to challenge the Notice of Examination. Cf. 
Matter of Lillard vy. Ginsberg, supra. Finally, the Ageney 
reviewed the Notice of Examination (Tr. 200) and ap- 
parently had no objection. 


The District Court cautions against over-estimating Ros- 
enberg’s prior experience since it occurred primarily in 
the area of classification. Classification or desk audits 
can serve as the basis for an examination (Tr. 230-231; 
431). Common sense tells us that the specific duties of 
a position to be described for purposes of classification 
do not change merely because the purpose is to prepare 
a-job analysis. The difference between describing the 
duties and responsibilities of a position for purposes of 
classification, on the one hand, and test preparation, on 
the other, is in singling out the specifie knowledge or 
abilities to be tested for, and weighting them properly. 


Cf. Kirkland v. N.Y. State Dep’t of Correctional Services, 
supra, 374 }. Supp. at 1374. There is no question here 
but that defendants set out to do a job analysis in prep- 


aration for an exam. 


The District Court counts onl ‘our permanent Sup 
I{R’s whom Rosenbere interviewed. The fact is that Ros 
enberg spoke to eight individuals wifh practical working 
knowledge of the agi nev. One of them, Howard Matelli, 
had been involved with setting up the HRA occupational 
series from the start of the ageney (123a).  Super- 
visors are an obvious source of information about the 
duties and responsibilities of lower positions and eon- 


ting supervisors is standard professional practice (Tr. 


While it may have been desirable to have a larger 


sampling of incumbents (Tr. 433-34), the knowledge eainod 
from the supervisors should not have been subtracted 
from Rosenbergs knowledge of the position as if it did 
not count. The purpose of the Court's inquiry into the 


method of preparation of the exam is not a mathematical 
exercise, If there were SO provisionals instead of 100, 
Rosenberg’s interviews of S persons with knowledge of 
the position of Sup HRS, in combination with his own 
working knowledge, would have been sufficient Ix yond 
eavil. It might well have been sufficient for an outsider 
called in to do a job analysis. 


The District Judge evaluated the testimony of defend- 
ants’ experts in light of his belief that the sum total of 
Rosenberg’s knowledee about the duties of a Sup HRS 
derived from four individuals out of a frroup of 180, or 
a 2% sample. In facet, defendants’ experts gave high 
marks to his efforts. Dr. Katzell testified that the job 
analysis (Tr. 409): 


“ 


was much better done than many with whieh 
[ had to work; it was more complete; the end 


product, the test plan were typical of test plans 
where knowledges and skills were categorized and 
proportional weights are listed to guide the per- 
son developing the examination.” 


The job analysis was “better than consistent” with pro- 
fessional standards (Tr. 409). 


As an added check, defendants asked Dr. Williams to 
make an inquiry parallel to that undertaken by Leonard 
Rosenberg in preparing the job analysis. Dr. Williams 
was examining the content validity procedure (Tr. 495; 
O04). He interviewed 12 different people over a period 
of 4% days (Tr. 496-497). He found the same variety 
of duties as did Mr. Rosenberg (Tr. 497): 


“ranging from people who had the responsibility 
of poliey and design . . . to people who did a lot 
of report writing ... to people who were involved 
with personnel and personnel review procedures ... 
There was a person in—I suppose I refer to it as 
the transportation distribution center where a lot 
of that kind of work was done—to people who were 
handling the low level budget.” 


Since Rosenberg was not going to test for a specifie 
intra-agency job, it is hard to see that interviewing these 
12 would have changed the inferences he made as to the 
skills to be tested. It might, however, have prevented 
the District Court’s getting bogged down on the sample 
size. Dr. Williams testified that, given the policy re- 
straints, “it was a very well done job of doing both the 
job analyses and the test construction. Under the eir- 
cumstances, I would say it was very good” (Tr. 498), 


[It is, of course, within the District Court’s diseretion 
to credit the testimony of the other side’s expert. How- 


ever, if it had eredited 
his words, the result in thi 
ly different. 


Dr. Lopez had, until recently, spent his career in pub 
lic personnel administration (560; 534: 680). Indeed, 
he had been extensively involved in preparing exams for 
the City of New York since about 1953 in the Personnel 
Examiner Series, and probably prepared the one Rosen 
berg took (Tr. 563, 651). Those exan nations tested for 
knowledge, and he testified it was appropriate to test for 
knowledge (Tr. 627: 654: 656). 


About four years ago, Dr. Lope; | his own busi- 
ness and now } : fi perso (Tr. 689). Where 
as he used st for knowledge, he now tests only fo1 
traits (Tr. 571; 644). He has tested for traits sueh 
as perception, memory, concentration, problem solving 
(Tr. 645) and decision-making ability (Tr. 579) which ap- 
parently was the only trait he measured for three dif- 
ferent levels of fire oflicers (Tr. 669). Some of the traits 
of a Sup HRS might be tolerance and influence (Tr. 


989)." Tle has supervised the preparation of about 200 


job analyses prepared by employers. He puts the in- 
dividuals who are to actually prepare the analyses through 
a training period lasting two or three days (Tr. 641). 
He has developed what he calls a “career matrix”, whieh 
took two hours to develop for the Ford Fou. .ation (Tr. 


642). 


Dr. Lopez is in a unique positi ¢ the business 


advantages of havine Courts ike knowledge ex- 


tified he was “glad” defendant id not 
try to test for ability to deal with O} “it is one of our more 
difficult problems which h not been ver ttectively solved.” 
(Tr. 437). 


45 


aminations in favor of trait tests (Tr. 630-631; 638). It 
will take perhaps a billion dollars nationwide to change 
the system (Tr. 638); the cost for validating one test 
for bus driver was $325,000 (Tr. 648), whereas Dr. Lopez 
believes that for the job “the only test you need is the 


possession of a driver’s license” (Tr. 676). 


When Dr. Lopez used to prepare tests for the City in 
the Personnel Examiner Series, he expected the eandi- 
dates “to have certain basie knowledge, be aware of cer- 
tain literature on the subject, and have done some readin: 
in these areas” (Tr. 654). As the Distriet Court pointed 
out, four years earlier “God had not illuminated his 
mind” to the extent of testing only for traits (Tr. 659) 


One is remingled of Judge Friendly’s words in Vulcan, 


supra, 490 F. 2d at 394: 


“The Fourteenth Amendment no more enacted a 
particular theory of psychological testing than it 


did Mr. Herbert Spencer’s Social Statics. Experi- 


ence teaches that the preferred method of today 
may be the rejected one of tomorrow.” 


The District Court found that Dr. Lopez’s criticisms 
of the job analysis were “sensible and persuasive”.* It 
relied on his testimony that the knowledge and skills were 
too ambiguous to give any real idea about what the job 
involves and that | pez would be unable to construct 
a content-valid test on the basis of the job analysis and 
test plan (353a). However, when Dr. Lopez prepared the 
examination for Personnel Examiner (Tr. 651), there was 
no “job analysis” at all (Tr. 652). Dr. Lopez worked ex- 
clusively frovy the “test plan” similar to the one pre- 


agreed to testify for plaintiffs before he had seen 
*r the job analysis or the test plan (Tr. 631-632). 
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pared for the exams in issue (Tr. 
satisfied with the test plan (Tr 


content of the written exam from 


Pr sumably Dr. Lopez d a need hea ‘tails from 
a job analysis to cloud his mind. t 1 vy les : Dr. 


Lopez’s testimony ndicat sufficiency of the 
sample size in a jol $1 iould not be over-empha 
sized if the areas t ; ste lor are clearly defined 
(Tr. 652). 


The other objections which Dr. Lopez raised, and thi 
District Court adopt d, are, submit, not “legitimate.” 
Vulcan, supra, 4990 F. 2d at 393, “or example, defend 
ants’ failure to indicate a “le f proficiency” is branded 
a “eritieal defect”. Tf it a defect, it is hardly eritical. 


Dr. Lopez means by vel of proficiency “knowledge that 
would ordinarily be “l by ft vears of training” 
(Tr. 516-517). This eriticism would be better taken for 
an outsider engaged to construct one exam. Here, Helene 
Willingham knew that she ws ‘on vting exams on 
three different levels of difficul vy an we’ differed by 40 
questions (Tr. 319-820; 


The District Court also found as a “serious defect’? 
that the job analysis “does not explain how or why the 
skills in the test plan were weighted as they were 
or even suggest—apart from the examples ov typical tasks 
—that the jobs held by those in the title of Sup. HRS 
can be different from one another and require different 


) 


abilities” (353a). 


These objections are words without meaning. The 


weighting on the test plan speaks for itself. Tt is a mat 


a 
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of judgment on the part of the person who performed 
job analysis, and is, as the District Court had just 


observed, a “largely inferer ‘al” step. Indeed, there is 


no other way (Tr. 526). There was testimony as to the 
competence of Leonard Rosenberg (Tr. 405) but the Court 
was not overly eager to credit someone trained in el: 
sification in any respect (supra, fn. p. 36). There was, fur 
thermore, testimony as to how Rosenberg weighted the 
areas to be ted (Tr. 199-202; 408-409; 436). The Court 
srudgingly conceded that Rosenberg stated the correct 
procedure to be followed but having given this much, re 
tracts it by falling back on the “critically insufficient” 
sample. 


he “tasks observed through interviews” deseribed in 


the job analysis or the “examples of typical tasks” on 
the Notice of Examination show that these are tasks done 
by Supervising IRS's, not that all Supervising HRS’s 
must do all those things (Tr. 470). The test plan speei- 
fies the knowledges and skills that are sie to doing any 
of these tasks (Tr. 471). 


Helene Wilingham prepared the examinations in the 


ltuman Resources occupational series at issue here, as 
she had prepared 200 examinations previously. In part, 
the Court criticizes her for exercising independent judg 
mer, but the Court’s principal criticisms of her question 
writing technique are rooted in the deficiencies he hs 

already found in the job analysis, i.e., “the fact that the 
Job analysis and test plan needed refinement by Willing- 
ham suggests that they were inadequate to begin with” 


ner 


(oo/a). 


In fact, defendants’ experts thought the examinations 
were well constructed (Tr. 413; 422; 498, 503). The item 
analysis provides a method other than subjective judg- 
ment to check on the item construction. High scoring 
‘andidates tended consistently to select the right answer,* 


* One question was a “negative indicator’; i.e., the high scor- 
ing candidates answered incorrectly. However, it was a good ques- 


n on Supervision (Tr. 420). 


tending 
indicator’ 
options, thi 
by itself evidence as to tl id pac if the content 
the questions (Tr. 417). 
The leneth to which 

finding that defendants could right can he seen 
in its treatment of defendants’ free training course to 
prepare HRA employees r the examination. Twenty 


questions, accounting for 25% of the examination, came 


from the training a Tr. 279). Defendants could 


not have done much better shor ‘issuing the questions 
in advance. ut th istrict rt manages to discount 
he value of this course Db tatine that “there is no evi- 


denee as to how many people 


‘he Court further disparages defendants’ training 
course for specifically preparing candid on graph in- 
terpretation and readn seems to imply that efforts 
to prepare candidates in this area were in less than 200d 
faith since ‘lene Willingham herself was sensitive to 
the fact that inoritie : ss wel mm these questions. 
Ilowever, tag such questions as having a “diserimina- 
tory bias” assumes the point to be proved. Whatever 
the disparate impact, the issue is whether the questions 
were job-related (Tr. 2! an issue e District Court 
never reached. Tn fact, the test plan called for such 
questions (195af1) and Helene Willinctam would have 
exceeded her tv had she eliminated them entirely. 
Furthermore, the direct { training at HRA had been 
associated with the ag y for vears and therefore 
“had a good irtea of » knowledges and skills that would 
be required for doing a ad ate job” (Tr. 280-281). 
Dr. Katzell found these questi “quite typical of the 
kinds of questions one finds in many national tests” (Tr. 


413). 


Testing is an imperfect science (Tr. 449-450) and there 
is no such thing as a perfect test (Tr. 485). Our candid 
admission that defendants were less than perfect in item 
construction (Tr, 411-412)* is neither devastating to their 
case nor inconsistent with the major premise that the ex- 
amination was, on the whole, well constructed to test for 
basic knowledge of the type that any Supervising Human 


Resources specialist ought to have to function effectively. 


It is our contention that defendants’ method of prepara- 
tion was consistent with professionally acceptable stand- 
ards of test preparation and that they were entitled to 
the benefit of the presumption, rebuttable to be sure, that 
snecess had been achieved. Walean. s pra, 490 F. 2d at 
396. The District Court, in finding otherwise, was clearly 
erroneous 


In focusing his attention t exclusively on allered 
imperfections in test preparation, practically as an in- 
tellectual exercise, the District Court lost sieht of the 
main duty at hand, to evaluate the examinations them- 
selves. That error is, we submi, fundamental and fatel. 


* Ambiguities in item construction are fertile ground for Acosta- 
type challenges, an explanation of why some “rules” are violated 
(Tr. 412). Acosta v. Lang, 13 N.Y.2d 1079 (1963). The Acosta 
challenge is the answer to Lopez's testimony, relied on by the 
District Court, that the test was poorly constructed “b--aiuse many 
questions appeared to have more than one correct aiswer, even 
to an expert.” In fact, there were only about 11 challenges to ques- 
tions (304), which Lopez later testified was evidence of the accep- 
tability of the questions to the candidates (679). As to the Court’s 
statement that the examination favored those with formal educa- 
tion, who 


educated ( 


test-wise, plaintiffs’ witnesses were both formally 


are 
Za: 


80; 107; 157) and presumably test-wise. 
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POINT III 


The challenged examinations were job-related in 
that they tested for the basic knowledge and skills 
necessary to perform a variety of duties as Human 
Rescurces Specialists. The District Court was clearly 
erroneous in finding that the examinations were not 
job-related without evaluating the purpose for which 
the exams were given and the examinations them- 
selves. 


The main thrust of defendants’ proof at trial was that 
we were testing for sufficient knowledge so that a person 
appointed to the position of Sup HRS was eapable of 
performing whatever job in the agency at the same level 
of responsibility and importance to which the individual 
inight be transferred (Tr. 14-15, 26, 28.29, 43; 93-94; 13: 


) 
133. 142: 297). This is consistent with a new over-all 
policy of “broad-banding”, to consolidate and pare down 
the 3,000 job titles in the City’s personnel system so that 
workers can be grouped as to equivalent occupational 
status to facilitate more flexibility in job assignment. “A 
common title will be given, for example, to relocation 
aides, public health assistants, and community liaison 
trainees, all of whom now earn the same salary and per- 
form comparable duties”. N.Y. Times, April 19, 19%, 


p. 34. 


The eases at bar, then, are particularly important to 
defendants. If defendants are required to test for spe- 
cific jobs “in twenty different kinds of programs” within 
one agency (349a), they are doomed before they begin 
to test for titles that span different agencies. 


Defendants had clustered the varying duties around 
eight areas of common knowledge or skills. The ¢xami- 
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nations themselves were before the Court (195ab3- 
19)ad21). For the Sup HRS exam, 25 questions related to 
the functions of HRA or other public agencies and cur. 
rent developments and problems in the field of human 
services (Tr. 278). Questions 1 to 25 related to super- 
vision and judgment in on-the-job situations (Tr. 277). 
Fifteen questions, Nos. 26 to 40, related to community 
relations and organization, and an additional 15 related 
to language usage and interpretations of tables and graphs 


QO 


(Tr. 277). The test was considered easy (Tr. 325). 


The purport of Dr. Katzell’s testimony was that de- 
fendants tested for what they said they were going to test 
for on the job analysis: the questions related to the test 
plan; the test plan related to the knowledge and _ skills 
required, which related to the tasks observed through 
interviews (Tr. 427). To thut extent, the test was job- 
related, and performance on the test could be expected 
to relate to performance on the job (see 206a). 


Harold Yourman, who had been with the Ageney vir- 
tually from its inception (Tr. 334) and who was familiar 
with the functions of the ageney (Tr. 352) and the title 
structure (Tr. 538; 342), was in a good position to evalu- 
ate whether the examinations were “content valid’. He 
testified that in his opinion the examination for Sup HRS 


was job-related to the variety of duties and responsibil- 
ities of the positir 1 (Tr. 363; 342; 354-355). 


In his former title as a Sup HRS, Yourman was re- 
sponsible for labor relations within HRA (Tr. 379). The 
District Judge stated that Yourman “conceded that the 
exam was not directly related to his earlier duties as a 
provisional Sup HRS.” The District Judge apparently 
believed that Yourman should have been tested on Jabor 
relations. Yourman testified that on the challenged ex- 
amination, “there was nothing specific in labor relations, 
but I would be responsible in my area to know about 
other areas in the Agency” (Tr. 395). 


Plaintiff Jones was also working as a Sup HRS in 
labor relations (Tr. 45). There was not a single other 
Sup HRS doing the same type of work (Tr. 47). Ques- 
tions in the area of labor relations, Jones testified, would 
be “totally baffling to anyone who hasn’t had some ex- 
perience in the area’ (T~ 53). 


Indeed, far from rebutting defendants’ ease, the plain- 
tiffs’ testimony at trial went far in the other direction 
towards proving it. ach witness testified to widely 
varying duties and responsibilities in a range of Agency 
positions. They then testified that the examinations were 
not job-ralated, as if defendants should have tested for 
the specific duties of, for example, Parent Coordinator 
in the Agency of Child Development (Tr. 101) or <As- 
sistant to the Deputy Director of Labor Relations (Tr. 
45) or Education Specialist in the Community Develop- 
ment Agency (Tr. 152; 158-162). 


From its appearance under subdivision “F” in the 
Opinion, the District Court treated det pee main de- 
fense as if it were peripheral to the issue to be decided. 
He had already decided against them on’ the issue of 
job-relatedness, purporvedly on the basis of “opinion tes- 
timony as to ane content validity of the exam itself” 
The content validity of an exam, |owever, is not an in- 
tellectual concept to be decided by the Court in a vacuum 


by sumply choosing Re pre conflicting opinion testimony. 


It is a factual determination: did the examinations prop- 
erly test for the duties aie responsibilities of the posi- 
tion? It ean only be decided by considering the examina- 
tions in light of the position being tested for. Since the 
District Court had made up its mind against defendants 
on the basis of the job-analysis and the “opinion testi- 
mony”, it did not bother to evaluate the examinations. 
This is squarely at variance with what this Court has 


only recently indicated as the correct approach: “the 
trial Judge could not confine himself to an examination 


of the process of preparation while completely ignoring 
the merit of the result.” Nirkland vy. N.Y. State De p’t of 
Correctional Services, supra, slip. op. at 5405. 


While, based on the approach here adopted by the Dis- 
trict Court, it is not surprising that the Court gave short 
shrift to defendants’ argument that they were testing for 
sufficient knowledge of a position to permit appointment 
to a wide variety of jobs and not for any individual job, 
the extent of his misunderstanding is revealing. The 
District Court cites as an “obvious example” of failure 
to have established a common core of skills the fact that 
20-25% of the questions on the Supervising HRS exam 
related to supervision when only 60% to 65% of the em- 
ployees have supervisory responsibility. The defect is 
apparently that those provisionals who currently do not 
have supervisory responsibility should n-t be tested on 
supervision. 


The Distriet Court might have referred to Dr. Lopez 
for an explanation as to why these questions were given. 
Dr. Lopez testified that it would be proper to test pros- 
pective Personnel Examiners on how to prepare a job 
analysis even though they did not in fact prepare job 
analyses (Tr. 654). The reason he gave was that once 
on the list (Tr. 655) 


“they could fill any vacancy that came up... We 
weren't testing for what he might do [in fact]. 
We felt that regardless of whether you were in 
training, classification, examining, there is certain 
knowledge about these other disciplines or the sub- 
disciplines that you should possess. A good per- 
sonnel man in the [selection] field knows classifi- 
cation and compensation ... He would have to 


know the purposes of a job analyses, and also be 
familiar with the various ways of conducting it 
. . I would expect them to have basie knowl- 


edge...” 


Dr. Lopez would expect the same basie knowledge, in 
his field, of a Supervising Human Resources Specialist 
(Tr. 656). In fact, Dr. Lopez testified that the Sup HRS 
test measured knowledge (Tr. 605). As for the questions 
on supervision, Dr. Lopez considered himself informed 
enough in that area to test himself (Tr. 591; 695; 651). 
He got 24 right out of 25, which is at least some evi- 
dence that the questions tested for what they were sup- 
possed to test. Plaintiff Jones testified that the questions 
on supervision were relevant to all the jobs in the title 


(Tr. 65-68 : see also SY: QS). 


The questions on the Sup HRS promotional exam, 
dealing with the functions of IMRA and current develop- 
ments and problems in the field of human services, pro- 
vide another example of the inadequacies of the Court’s 


analysis. Plaintiffs testified that the position of a Super- 


ising HRS is a professional and very responsible posi- 


ion, which might entail the interpretation and application 
federal laws and programs (Tr. 50; 128-129, see Tr. 


39). They testified that it would be important to know 


*hanges in federal law which might affect one of the pro- 
grams run by HRA (Tr. 129). Since the Agency deals 
with low-income peojie who are likely to need a variety 
of services from the City, it would be important to know 
what programs the Agency offers (Tr. 130; 141-142). 


The only comment the District Court makes on this 
area is that 


“Tt is difficult to see how such questi ‘an be 
considered essential to all those in the tiu. in view 


of the fact that the open competitive exam for 
Sup HRS omitted these questions in favor of more 
general questions dealing with ‘Functions of Rele- 
vant Public and Private Agencies.’ 


Putting aside the fact that in a civil service jurisdic- 
tion the promotional candidates are customarily favored 
and there would be no rationale for appointing them first 
if the examinations for the same position were identical, 
the District Court misses the point. The correct inquiry 
is whether the functions of the agencies dealing in the 
anti-poverty and social welfare areas are relevant areas 
of knowledge on which to test individuals to be employed 
in the Agency, not whether individual questions which 
test these areas are identical. Cf. Vulcan v. Civil Service 
Comm'n, supra, 490 F. 2d at 395. The District Court 
never addressed itself to this issue. 


f 
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The District Judge concludes that we could not have 
been successful in testing for a common core of knowl- 
edge and skills because substantial numbers of provi- 
sionals failed, although their performance had been rated 
as satisfactory.* This simply states the case before the 
Court. It was the Court's duty to evaluate the examina- 
tions—which he did not do—in light of the purpose for 
which they were given, a purpose to which he had closed 
his mind. 


When all is said and done, defendants are faulted for 
not giving a predictively validated test which, as the Dis- 

* The impressions of two supervisors should not be given undue 
weight in the absence of objective performance criteria (Tr. 531- 
#32; 541-543). See Albermarle Paper Co. v. Moody, US. 
—, 45 (1975) L. Ed. 2d 289, 305. The bulk of the provisionals 
were evaluated on a gross satisfactory-unsatisfactory scale (Tr. 
557). It is typical that individuals get satisfactory ratings on such 
a scale (Tr. 455-456). 


trict Court viewed it, is the only reason for giving an 
examination (359a). Most of the transcript references 
in support of the Court’s statement refer to Dr. Lopez, 
who today tests only for traits, but vesterday tested for 
knowledge and may do so tomorrow. Ilowever, defend 
ants did not pretend to have given a test which empiri- 
eally predicted suecess (Tr. 540-546). As in any knowl- 
edge test, the inference is made that if the knowledge is 
eit it will be usefully applied (Tr. 488). It is hard 
to see why testing for traits promises a better result. 
A fire officer may have commendable decision making abil- 


ity. = might make the wrong decision at a fire, espe- 


cially » has been promoted solely on the basis of his 
trait, ds not on the basis rae knowledge of fire-fighting 
techniques. As the Court ‘ here noted, “the eritical 


question is whether the cea ake procedure is con: *itu- 
tionally sound, not whether a better one could have ween 


devised”? (326a). 


It is defendants’ position that they were entitled to a 
rebuttable presumption that the examinations were job- 
related. Even without a favorable presumption working 
in their favor, they proved that the examinations tested 
for sufficient basic knowledge to permit persuns appointed 
to the positions of Iluman Resources Specialist to work 
in a variety of jobs throughout the Agency. Plaintiffs 
themselves attested to the variety. It was reversible 
error for the Court to have found that the examinations 
were not job-related witlout evaluating the purpose for 
which they were given and, in light of such purposes, the 
examinations themselves. 


CONCLUSION 


The order and judgment appealed from should be 
reversed and the complaints dismissed. 


August 27, 1975 
Respectfully submitted, 


W. Bernarp RicHuanp, 
Corporation Counsel, 
Attorney for Defendants-Appellants. 


L. Kevin SHERIDAN, 
Pauta J. OMANSEY, 
of Counsel. 


ADDENDUM 


Tables I and II 


[PHorostats | 


(Opposite) Sa 


Challenged Exam No. 1631 (Sup. HRS) (Prom.) 


Pass Fail Total % Passing 


Blacks 12 69 17% 
Whites 28 52 54% 
Hispanics 3 16 19% 
Unknown _ 1 


—_—— —_—— 


43 138 


Challenged Exam No. 2013 (Sup. HRS) (OC) 


Pail Total % Passing 


Blacks 208 247 16% 

Whites 108 233 54% 

Hispanics 17 20 15% 

Athers 3 63% 
Subtotal 336 

Unknown 183 303 38% 
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Challenged Exam No. 1626 (Sr. HRS) (Prom.) 


Challenged Exam No. i¢} °°: 
Passed Failed Total % Passing 


Blacks 11 51 62 18% 
Whites 30 4 34 88% 
Hispanic 37% 
Other 


44 


Challenged Exam No. 1099 (Sr. HRS) (OC) 


Blacks 56 165 221 2u% 

Whites 101 54 155 65% 

Hispanic 8 22 30 27% 
Subtotal 165 241 406 

Unknown 90 187 277 32% 


255 428 683 
Challenged Exam No. 1097 (HRS) (OC) 


Blacks 55 120 175 
Whites 59 56 115 
Hispanics 29 36 
Other 

Subtotal 
Unknown 


TABLE II 


HRS Promotional Exam No. 1625 
Passed Failed Total % Passing 


Blacks 36 27% 
Whites 4 20% 
Hispanics 4 


— 2 
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Sr. HRS (MDT) Open Competitive Exam No. 1094 
Passed Failed Total % Passing 


— 


Blacks 18 31% 
Whites 33% 


Other 


5 
Hispanics 7 32% 


Subtotal 0 
Unknown 8 


“ 193 


HRS (MDT) Open Competitive Fxam No. 1095 
Passed Failed Total 


Black 56 21%, 
White 17 47%, 
Hispanic 23 30% 


Subtotal 
No ethnic 
info 


Pl 


